Mesoscopic Biochemical Basis of Isogenetic Inheritance and Canalization: Stochasticity, Nonlinearity, and Emergent Landscape

Biochemical reaction systems in mesoscopic volume, under sustained environmental chemical gradient(s), can have multiple stochastic attractors. Two distinct mechanisms are known for their origins: ($a$) Stochastic single-molecule events, such as gene expression, with slow gene on-off dynamics; and ($b$) nonlinear networks with feedbacks. These two mechanisms yield different volume dependence for the sojourn time of an attractor. As in the classic Arrhenius theory for temperature dependent transition rates, a landscape perspective provides a natural framework for the system's behavior. However, due to the nonequilibrium nature of the open chemical systems, the landscape, and the attractors it represents, are all themselves {\em emergent properties} of complex, mesoscopic dynamics. In terms of the landscape, we show a generalization of Kramers' approach is possible to provide a rate theory. The emergence of attractors is a form of self-organization in the mesoscopic system; stochastic attractors in biochemical systems such as gene regulation and cellular signaling are naturally inheritable via cell division. Delbr\"{u}ck-Gillespie's mesoscopic reaction system theory, therefore, provides a biochemical basis for spontaneous isogenetic switching and canalization.Comment: 24 pages, 6 figure

Mesoscopic Kinetic Basis of Macroscopic Chemical Thermodynamics: A Mathematical Theory

From a mathematical model that describes a complex chemical kinetic system of $N$ species and $M$ elementrary reactions in a rapidly stirred vessel of size $V$ as a Markov process, we show that a macroscopic chemical thermodynamics emerges as $V\rightarrow\infty$. The theory is applicable to linear and nonlinear reactions, closed systems reaching chemical equilibrium, or open, driven systems approaching to nonequilibrium steady states. A generalized mesoscopic free energy gives rise to a macroscopic chemical energy function \varphi^{ss}(\vx) where \vx=(x_1,\cdots,x_N) are the concentrations of the $N$ chemical species. The macroscopic chemical dynamics \vx(t) satisfies two emergent laws: (1) (\rd/\rd t)\varphi^{ss}[\vx(t)]\le 0, and (2)(\rd/\rd t)\varphi^{ss}[\vx(t)]=\text{cmf}(\vx)-\sigma(\vx) where entropy production rate $\sigma\ge 0$ represents the sink for the chemical energy, and chemical motive force $\text{cmf}\ge 0$ is non-zero if the system is driven under a sustained nonequilibrium chemostat. For systems with detailed balance $\text{cmf}=0$, and if one assumes the law of mass action,\varphi^{ss}(\vx) is precisely the Gibbs' function $\sum_{i=1}^N x_i\big[\mu_i^o+\ln x_i\big]$ for ideal solutions. For a class of kinetic systems called complex balanced, which include many nonlinear systems as well as many simple open, driven chemical systems, the \varphi^{ss}(\vx), with global minimum at \vx^*, has the generic form $\sum_{i=1}^N x_i\big[\ln(x_i/x_i^*)-x_i+x_i^*\big]$,which has been known in chemical kinetic literature.Macroscopic emergent "laws" are independent of the details of the underlying kinetics. This theory provides a concrete example from chemistry showing how a dynamic macroscopic law can emerge from the kinetics at a level below.Comment: 8 page

Landscapes of Non-gradient Dynamics Without Detailed Balance: Stable Limit Cycles and Multiple Attractors

Landscape is one of the key notions in literature on biological processes and physics of complex systems with both deterministic and stochastic dynamics. The large deviation theory (LDT) provides a possible mathematical basis for the scientists' intuition. In terms of Freidlin-Wentzell's LDT, we discuss explicitly two issues in singularly perturbed stationary diffusion processes arisen from nonlinear differential equations: (1) For a process whose corresponding ordinary differential equation has a stable limit cycle, the stationary solution exhibits a clear separation of time scales: an exponential terms and an algebraic prefactor. The large deviation rate function attains its minimum zero on the entire stable limit cycle, while the leading term of the prefactor is inversely proportional to the velocity of the non-uniform periodic oscillation on the cycle. (2) For dynamics with multiple stable fixed points and saddles, there is in general a breakdown of detailed balance among the corresponding attractors. Two landscapes, a local and a global, arise in LDT, and a Markov jumping process with cycle flux emerges in the low-noise limit. A local landscape is pertinent to the transition rates between neighboring stable fixed points; and the global landscape defines a nonequilibrium steady state. There would be nondifferentiable points in the latter for a stationary dynamics with cycle flux. LDT serving as the mathematical foundation for emergent landscapes deserves further investigations.Comment: 4 figur

Analytical Mechanics in Stochastic Dynamics: Most Probable Path, Large-Deviation Rate Function and Hamilton-Jacobi Equation

Analytical (rational) mechanics is the mathematical structure of Newtonian deterministic dynamics developed by D'Alembert, Langrange, Hamilton, Jacobi, and many other luminaries of applied mathematics. Diffusion as a stochastic process of an overdamped individual particle immersed in a fluid, initiated by Einstein, Smoluchowski, Langevin and Wiener, has no momentum since its path is nowhere differentiable. In this exposition, we illustrate how analytical mechanics arises in stochastic dynamics from a randomly perturbed ordinary differential equation $dX_t=b(X_t)dt+\epsilon dW_t$ where $W_t$ is a Brownian motion. In the limit of vanishingly small $\epsilon$, the solution to the stochastic differential equation other than $\dot{x}=b(x)$ are all rare events. However, conditioned on an occurence of such an event, the most probable trajectory of the stochastic motion is the solution to Lagrangian mechanics with $\mathcal{L}=\|\dot{q}-b(q)\|^2/4$ and Hamiltonian equations with $H(p,q)=\|p\|^2+b(q)\cdot p$. Hamiltonian conservation law implies that the most probable trajectory for a "rare" event has a uniform "excess kinetic energy" along its path. Rare events can also be characterized by the principle of large deviations which expresses the probability density function for $X_t$ as $f(x,t)=e^{-u(x,t)/\epsilon}$, where $u(x,t)$ is called a large-deviation rate function which satisfies the corresponding Hamilton-Jacobi equation. An irreversible diffusion process with $\nabla\times b\neq 0$ corresponds to a Newtonian system with a Lorentz force $\ddot{q}=(\nabla\times b)\times \dot{q}+1/2\nabla\|b\|^2$. The connection between stochastic motion and analytical mechanics can be explored in terms of various techniques of applied mathematics, for example, singular perturbations, viscosity solutions, and integrable systems.Comment: 23 pages, 2 figure

Interaction-aware Factorization Machines for Recommender Systems

Factorization Machine (FM) is a widely used supervised learning approach by effectively modeling of feature interactions. Despite the successful application of FM and its many deep learning variants, treating every feature interaction fairly may degrade the performance. For example, the interactions of a useless feature may introduce noises; the importance of a feature may also differ when interacting with different features. In this work, we propose a novel model named \emph{Interaction-aware Factorization Machine} (IFM) by introducing Interaction-Aware Mechanism (IAM), which comprises the \emph{feature aspect} and the \emph{field aspect}, to learn flexible interactions on two levels. The feature aspect learns feature interaction importance via an attention network while the field aspect learns the feature interaction effect as a parametric similarity of the feature interaction vector and the corresponding field interaction prototype. IFM introduces more structured control and learns feature interaction importance in a stratified manner, which allows for more leverage in tweaking the interactions on both feature-wise and field-wise levels. Besides, we give a more generalized architecture and propose Interaction-aware Neural Network (INN) and DeepIFM to capture higher-order interactions. To further improve both the performance and efficiency of IFM, a sampling scheme is developed to select interactions based on the field aspect importance. The experimental results from two well-known datasets show the superiority of the proposed models over the state-of-the-art methods